Convergence to Global Optimality with Sequential Bayesian Sampling Policies

We consider Bayesian information collection, in which a measurement policy collects information to support a future decision. This framework includes problems in ranking and selection, reinforcement learning, and continuous global optimization. We give sufficient conditions under which measurement policies achieve asymptotically minimal expected loss. Achieving asymptotically minimal expected loss implies that the sequence of decisions believed to be the best under successive posterior distributions converges almost surely to the set of globally optimal decisions. This condition is most useful for adaptive sequential sampling policies, which often perform better than nonadaptive policies, but whose convergence is often difficult to confirm by other means. We apply these sufficient conditions to show convergence to global optimality for three previously proposed ranking and selection policies: OCBA for linear loss, LL(S), and LL1. We also show how this sufficient condition may be applied to knowledge-gradient policies.